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Grant  Title:  Provable  Security  of  Communication  for  Protecting  information  Flow  in 
Distributed  Systems 

Grant#:  FA9550-12-1-0196 

Principle  Investigator:  Dr.  Paul  Cuff 

Reporting  Period:  1  July  2012  to  30  June  2015 


Abstract: 

This  report  includes  a  detailed  summary  of  the  highlights  of  the  research  project  during  the 
final  year  of  funding  as  well  as  a  complete  report  on  the  publications  and  results  produced 
during  the  entire  duration  of  the  project.  There  are  also  three  research  papers  under 
review  and  two  in  preparation,  which  are  not  listed  in  this  report. 


Report  for  final  year: 

Annual  accomplishments  (final  one  year): 

Secret  key  agreement  with  Gaussian  sources:  Complementary  to  this  project  is  the 
production  of  secure  digital  resources  that  are  used  for  secure  encoding  of  signals.  Within 
this  space  is  physical  layer  security  and,  in  particular,  secret  key  agreement.  The  secret  key 
agreement  problem  where  correlated  observations  are  used  to  generate  a  key,  with  the  use 
of  limited  public  communication,  has  been  solved  in  the  literature  for  two  nodes  with  i.i.d. 
sources,  to  the  point  of  an  information  theoretic  statement  that  requires 
optimization.  Explicit  formulas  have  been  derived  in  the  literature  for  the  important 
Gaussian  case.  We  have  provided  an  explicit  solution  for  vector  Gaussian  sources  and 
extended  to  arbitrary  stationary  Gaussian  sources.  This  is  one  of  the  most  relevant  cases  in 
practice,  and  the  solution  has  a  nice  water-filling  interpretation  over  the 
spectrum.  Furthermore,  the  analysis  provides  new  insights  into  the  tensorization  of 
extremal  mutual  information  quantities,  which  have  been  of  recent  interest  in  the 
community.  Additionally,  we  have  derived  fundamental  limits  for  secret  key  agreement 
among  more  than  two  nodes  with  a  single  communicator,  for  general  distributions.  This 
has  been  submitted  as  a  journal  paper. 

Secure  source  coding  with  side  information  or  a  helper:  We  developed  schemes  for 
utilizing  side-information  in  a  secure  signal  encoding  protocol.  Two  settings  have  been 
studied— one  where  the  side-information  is  at  the  decoder,  and  another  where  a  helper 
encodes  the  side-information,  in  some  situations  (with  no  causal  disclosure),  the  side- 


information  can  allow  us  to  completely  obscure  the  signal  from  the  eavesdropper. 

Joint  source-channel  coding:  While  most  of  this  project  deals  with  secure  source  encoding 
(compression),  we  have  shown  that  if  source  coding  and  channel  coding  are  designed 
jointly  and  used  to  transmit  information  across  a  noisy  channel  they  can  perform  better 
than  separate  designs  connected  together.  This  differs  from  the  source-channel  separation 
theorem  for  point-to-point  non-secure  communication. 

Common  information  for  Gaussian  variables:  Wyner  defined  an  important  information 
quantity  called  common  information  which  plays  an  important  role  in  secure 
compression.  Recently,  an  explicit  formula  for  the  scalar  Gaussian  common  information 
was  derived  in  the  literature.  We  provided  the  vector  Gaussian  formula.  The  proof 
technique  extends  to  more  complicated  information  theoretic  regions  than  common 
information  (such  as  those  in  our  theorem  statements). 

Differential  Privacy  and  Mutual  Information:  A  recent  popular  notion  of  database  privacy 
is  "differential  privacy."  We  have  discovered  a  fundamental  connection  between  this 
metric  of  privacy  and  a  mutual  information  quantity.  This  connection  allows  a  deep 
understanding  of  what  the  metric  is  actually  assuring. 


Professional  Human  Training  Opportunities  (final  one  year): 

Curt  Schieler  completed  his  Ph.D  thesis  and  degree  based  on  this  project. 

Two  women  have  played  major  roles  in  this  research  project  over  the  past  year. 

Seven  Ph.D  students  conducted  research  as  part  of  this  project. 

Ph.D.  students  C.  Song,  S.  Satpathy,  and  J.  Liu,  each  advised  by  the  P.I.,  attended  the  2014 
North  American  School  of  Information  Theory.  They  attended  lectures  and  presented 
posters  explaining  their  recent  contributions  toward  information  theoretic  secrecy  of 
signals. 

Three  undergraduates  conducted  research  projects  with  the  P.l.  Aaron  Himelman,  worked 
on  developing  a  new  ranking  algorithm  for  sports  teams.  Michael  Freyberger  worked  on 
zero-delay  source  coding  for  secrecy.  Timothy  Seah  implemented  a  novel  LASSO 
optimization  algorithm  for  pitch  and  instrument  detection.  Each  of  these  students  gave  a 
presentation  and  wrote  a  report  on  the  work. 


Disseminated  to  communities  of  interest  (final  one  year): 

Two  IEEE  journal  articles  have  been  published  in  the  last  year,  related  to  this 
project.  Three  more  journal  articles  from  this  work  are  under  review. 


The  P.I.  gave  a  three  hour  tutorial  on  the  topic  of  this  project  at  SPCOM  2014.  The  P.I.  has 
also  been  invited  to  give  four  other  conference  talks  on  the  topic. 

Additionally,  five  other  conference  presentations  on  the  topic  of  this  project  have  been 
given  by  Ph.D.  students  during  the  past  year. 


Impact  on  the  development  of  the  principal  discipline  of  the  project  (final  one  year): 

The  primary  way  that  the  fields  of  security  and  information  theory  are  affected  by  the 
recent  developments  of  this  project  is  by  filling  in  a  missing  piece  of  crucial 
understanding.  We  can  roughly  divide  the  field  up  into  four  main  categories:  first,  channel 
coding  (the  conversion  of  noisy  and  imperfect  physical  resources  into  ideal  digital 
resources);  second,  source  coding  (the  conversion  of  information  signals  into  digital 
representations);  and  third  and  fourth,  the  secure  versions  of  the  first  two.  This  project 
adds  crucial  understanding  to  the  fourth  category,  secure  source  coding.  This  work  is  very 
different  from  previous  approaches  in  the  field,  and  the  P.I.  expects  that  this  project  may 
ultimately  be  viewed  as  the  right  way  to  approach  this  problem.  For  one  thing,  we  have 
shown  this  approach  to  be  a  generalization  of  the  standard  previous  approach. 

This  work  also  contributes  the  likelihood  encoder  technique,  which  is  a  generally  useful  in 
the  field  of  information  theory.  It  is  already  beginning  to  be  used  by  others  in  the  field. 

Secret  key  agreement  has  been  an  important  and  well  studied  topic.  Our  solution  for  the 
stationary  Guassian  case  brings  this  closer  to  practical  applications. 


Impact  on  other  disciplines  (final  one  year): 

The  simplified  understanding  of  causal  secrecy  as  synthesizing  a  memoryless  channel 
(published  in  a  journal  article  this  year)  puts  the  technology  in  a  position  to  be  utilized  for 
distributed  control.  We  have  begun  to  make  this  claim  concrete.  Potentially  this 
technology  will  become  a  staple  for  efficiently  using  digital  resources  in  a  secure  control 
system,  in  such  a  way  that  is  easy  to  analyze. 

Also,  in  the  field  of  control,  our  bounds  on  convergence  time  for  distributed  consensus  are 
the  leading  universal  bounds  for  this  actively  studied  problem. 

The  differential  privacy  discoveries  of  this  project  will  affect  the  way  differential  privacy  is 
understood  and  used  in  other  fields  such  as  computer  science,  statistics,  and  machine 
learning. 


Impact  on  society  beyond  science  and  technology  (final  one  year): 


Secure  encoding  of  information  enables  the  design  of  systems  of  great  importance  to 
society.  Upon  applying  this  theory,  important  infrastructure  used  ubiquitously,  such  as 
communication  networks,  power  grids,  etc.,  can  be  made  more  secure  again  malicious 
attacks. 

Archival  Publications  supported  by  this  project  (final  one  year— journal  articles  in  bold): 

1)  E.  Song,  P.  Cuff,  V.  Poor,  "Joint  Source-Channel  Secrecy  using  Hybrid  Coding,"  Proc.  of 
IEEE  Int’l.  Symp.  on  Inf.  Theory  (ISIT),  June,  2015. 

2)  S.  Satpathy  and  P.  Cuff,  "Gaussian  Secure  Source  Coding  and  Wyner's  Common 
Information,"  Proc.  of  IEEE  Int'l.  Symp.  on  Inf.  Theory  (ISIT),  June,  2015. 

3)  J.  Liu,  P.  Cuff,  S.  Verdu,  "Secrecy  Key  Generation  with  One  Communicator  and  a  One- 
Shot  Converse  via  Hypercontractivity,"  Proc.  of  IEEE  Int’l.  Symp.  on  Inf.  Theory 
(ISIT),  June,  2015. 

4)  J.  Liu,  P.  Cuff,  S.  Verdu,  "One-shot  Mutual  Covering  Lemma  and  Marton's  Inner  Bound 
with  a  Common  Message,"  Proc.  of  IEEE  Int’l.  Symp.  on  Inf.  Theory  (ISIT),  June,  2015. 

5)  J.  Liu,  P.  Cuff,  S.  Verdu,  "Resolvability  in  E-gamma  with  applications  to  Lossy 
Compression  and  Wiretap  Channels,"  Proc.  of  IEEE  Int’l.  Symp.  on  Inf.  Theory  (ISIT), 
June,  2015. 

6)  S.  Shang,  P.  Cuff,  P.  Hui,  S.  Kulkarni,  "An  Upper  Bound  on  the  Convergence 
Time  for  Quantized  Consensus  of  Arbitrary  Static  Graphs,"  IEEE  Trans,  on 
Automatic  Control,  60(4):1127-32,  April,  2015. 

7)  C.  Schieler  and  P.  Cuff,  "Rate-Distortion  Theory  for  Secrecy  Systems,"  IEEE 
Trans,  on  Inf.  Theory,  60(12) :7584-605,  Dec.,  2014. 

8)  E.  Song,  P.  Cuff,  V.  Poor,  "A  Rate-Distortion  Based  Secrecy  System  with  Side 
Information  at  the  Decoders,"  Proc.  of  the  Allerton  Conference  on  Communication, 
Control,  and  Computing,  Oct.  2014. 

9)  S.  Satpathy  and  P.  Cuff,  "Secure  Coordination  with  a  Two-Sided  Helper,"  Proc.  of 
IEEE  Int’l.  Symp.  on  Inf.  Theory  (ISIT),  July,  2014. 

10) E.  Song,  P.  Cuff,  V.  Poor,  "The  Likelihood  Encoder  for  Lossy  Source  Compression," 
Proc.  of  IEEE  Int’l.  Symp.  on  Inf.  Theory  (ISIT),  July,  2014. 

11) C.  Schieler  and  P.  Cuff,  "The  Henchman  Problem:  Measuring  Secrecy  by  the 
Minimum  Distortion  in  a  List,"  Proc.  of  IEEE  Int’l.  Symp.  on  Inf.  Theory  (ISIT),  July, 
2014. 

12) J.  Liu,  P.  Cuff,  S.  Verdu,  "Key  Capacity  with  Limited  One-Way  Communication  for 
Product  Sources,"  Proc.  of  IEEE  Int’l.  Symp.  on  Inf.  Theory  (ISIT),  July,  2014. 

13) S.  Shang,  Y.  Hui,  P.  Hui,  P.  Cuff,  S.  Kulkarni,  "The  Application  of  Differential  Privacy 
for  Rank  Aggregation:  Privacy  and  Accuracy,"  Proc.  of  the  Int’l.  Conf.  on  Information 
Fusion,  July,  2014. 


Complete  report: 


Research  highlights: 

The  journal  publications  produced  by  this  project  encapsulate  the  most  exciting  results  of 
this  project,  in  particular  7)  and  16)  below  (three  other  journal  papers  are  currently  under 
review).  These  articles  have  the  potential  to  become  groundwork  for  a  new  theory  of 
secure  source  coding.  Others  in  the  information  theory  field  are  beginning  to  take  interest. 
The  novelty  is  that  it  is  a  theory  about  compressing  and  encoding  information  in  a  secure 
way,  which  can  be  employed  at  the  end-points  of  the  network  rather  than  the  physical  layer 
if  desired,  it  differs  from  other  work  on  secure  source  coding  in  that  security  is  measured 
by  what  the  eavesdropper  can  do  with  the  obtained  information,  in  a  distortion  or 
competitive  sense.  The  content  of  7)  outlines  the  fundamental  tradeoffs  between  security 
resources  expended  and  performance.  We  were  able  to  develop  this  theory  with 
definitions  much  stronger  than  originally  anticipated.  Furthermore,  to  our  surprise,  we 
were  able  to  show  rigorously  that  this  is  a  richer  and  more  general  theory  than  the 
traditional  approach,  which  uses  entropy  as  a  measure  of  security  (referred  to  as 
"equivocation"). 

This  main  contribution,  described  in  the  previous  paragraph,  has  consequences  beyond  the 
field  of  information  theory  and  communication.  Since  the  theory  can  handle  signals  that 
come  from  sensors,  it  is  a  relevant  approach  for  encoding  signals  that  are  used  for 
distributed  control.  Thus,  the  project  outputs  are  bridging  across  fields  and  addressing 
challenges  in  control. 

The  article  16)  is  a  new  communication  tool  for  synthesizing  genuine  random  noise  at  a 
remote  location  that  is  correlated  with  local  information.  This  technique  plays  a 
fundamental  role  in  secure  encoding  of  information  (such  as  sensor  measurements),  it  is 
also  a  useful  tool  for  coordination  among  various  nodes.  This  result  has  influenced  the  field 
by  inspiring  several  follow  on  projects  by  other  groups  at  other  institutions. 

An  analysis  tool  emerged  from  this  research  project,  which  we  refer  to  as  the  "likelihood 
encoder."  We  have  spoken  about  this  tool  at  conferences  (10  and  18  below),  and  a  journal 
paper  is  under  review,  it  turns  out  to  be  broadly  applicable  for  source  coding.  This  tool 
allows  for  cleaner,  more  powerful  proofs  and  has  the  potential  to  broadly  impact  the 
techniques  used  in  information  theory. 
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